{"cells":[{"cell_type":"markdown","source":["[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/12QsnkUNOivLId1VP0IN00yYq99AA4UUk?usp=sharing)"],"metadata":{"id":"i8vYht2iBOB0"}},{"cell_type":"markdown","source":["<h1 align=\"center\">\n","  <a href=\"https://portkey.ai\">\n","    <img width=\"300\" src=\"https://analyticsindiamag.com/wp-content/uploads/2023/08/Logo-on-white-background.png\" alt=\"portkey\">\n","  </a>\n","</h1>"],"metadata":{"id":"wrkldbuofViJ"}},{"cell_type":"markdown","source":["# Portkey + OpenAI"],"metadata":{"id":"tO8z3C5Qe0i-"}},{"cell_type":"markdown","source":["[Portkey](https://app.portkey.ai/) is the Control Panel for AI apps. With it's popular AI Gateway and Observability Suite, hundreds of teams ship reliable, cost-efficient, and fast apps.\n","\n","With Portkey, you can\n","\n"," - Connect to 150+ models through a unified API,\n"," - View 40+ metrics & logs for all requests,\n"," - Enable semantic cache to reduce latency & costs,\n"," - Implement automatic retries & fallbacks for failed requests,\n","\n"],"metadata":{"id":"uHYmGi-Qb5eL"}},{"cell_type":"markdown","source":["You will need Portkey and OpenAIAI API keys to run this notebook.\n","\n","- Sign up for Portkey and generate your API key [here](https://app.portkey.ai/).\n","- Get your OpenAI API key [here](https://console.OpenAI.com/keys)"],"metadata":{"id":"l3OZLUkNcDfD"}},{"cell_type":"code","execution_count":2,"metadata":{"id":"7mzpClSpAgTW","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1715679908597,"user_tz":-330,"elapsed":15068,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}},"outputId":"61dda36e-4553-466e-de74-7a2dfb277bf9"},"outputs":[{"output_type":"stream","name":"stdout","text":["\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.3/62.3 kB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m320.3/320.3 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.7/12.7 MB\u001b[0m \u001b[31m69.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h"]}],"source":["!pip install -qU portkey-ai openai"]},{"cell_type":"markdown","metadata":{"id":"WacBHzekCXVf"},"source":["## With OpenAI Client"]},{"cell_type":"markdown","source":["### Quick Start"],"metadata":{"id":"u51oIoo18yBu"}},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9411,"status":"ok","timestamp":1715680013747,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"},"user_tz":-330},"id":"tvmQtgsWBur2","outputId":"61516ea2-d9fc-4606-a9c5-3af0a2ca9b7b"},"outputs":[{"output_type":"stream","name":"stdout","text":["A fractal is a complex geometric shape that can be split into parts, each of which is a reduced-scale copy of the whole. This property is known as self-similarity. Fractals are often found in nature, where many complex structures are similarly self-similar across different scales. Examples include snowflakes, mountain ranges, lightning bolts, and even some aspects of human anatomy, such as bronchial tubes and blood vessels.\n","\n","Mathematically, fractals can be described by iterative processes, where a simple geometric rule is repeatedly applied to generate patterns. Famous fractals include the Mandelbrot set, the Julia set, and the Sierpinski triangle. These shapes often have intricate, detailed structures at every level of magnification, making them fascinating both visually and mathematically.\n","\n","Fractals have applications in numerous fields such as computer graphics, natural sciences, and even finance, where they help model complex, irregular phenomena and systems.\n"]}],"source":["# Chat with GPT-4o\n","\n","from openai import OpenAI\n","from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n","from google.colab import userdata\n","\n","portkey = OpenAI(\n","    api_key= userdata.get('OPENAI_API_KEY'), ## replace it your OpenAI API key\n","    base_url=PORTKEY_GATEWAY_URL,\n","    default_headers=createHeaders(\n","        provider=\"openai\",\n","        api_key= userdata.get('PORTKEY_API_KEY'), ## replace it your Portkey API key\n","    )\n",")\n","\n","chat_complete = portkey.chat.completions.create(\n","    model=\"gpt-4o\",\n","    messages=[{\"role\": \"user\",\n","               \"content\": \"What is a fractal?\"}],\n",")\n","\n","print(chat_complete.choices[0].message.content)"]},{"cell_type":"markdown","source":["### Image Processing with GPT-4o"],"metadata":{"id":"vpUp3l3p81hP"}},{"cell_type":"code","source":["## URL Image Processing\n","\n","response = portkey.chat.completions.create(\n","    model=\"gpt-4o\",\n","    messages=[\n","        {\"role\": \"system\", \"content\": \"You are a helpful assistant that responds in Markdown. Help me with my math homework!\"},\n","        {\"role\": \"user\", \"content\": [\n","            {\"type\": \"text\", \"text\": \"What's the area of the triangle?\"},\n","            {\"type\": \"image_url\", \"image_url\": {\n","                \"url\": \"https://upload.wikimedia.org/wikipedia/commons/e/e2/The_Algebra_of_Mohammed_Ben_Musa_-_page_82b.png\"}\n","            }\n","        ]}\n","    ],\n","    temperature=0.0,\n",")\n","\n","print(response.choices[0].message.content)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"gbBK0Rpo8g9X","executionInfo":{"status":"ok","timestamp":1715680021631,"user_tz":-330,"elapsed":4482,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}},"outputId":"8a86ca57-c64b-4710-8926-c317d3af0765"},"execution_count":5,"outputs":[{"output_type":"stream","name":"stdout","text":["To find the area of the triangle, we can use Heron's formula. First, we need to find the semi-perimeter of the triangle.\n","\n","The sides of the triangle are 6, 5, and 9.\n","\n","1. Calculate the semi-perimeter \\( s \\):\n","\\[ s = \\frac{a + b + c}{2} = \\frac{6 + 5 + 9}{2} = 10 \\]\n","\n","2. Use Heron's formula to find the area \\( A \\):\n","\\[ A = \\sqrt{s(s-a)(s-b)(s-c)} \\]\n","\n","Substitute the values:\n","\\[ A = \\sqrt{10(10-6)(10-5)(10-9)} \\]\n","\\[ A = \\sqrt{10 \\cdot 4 \\cdot 5 \\cdot 1} \\]\n","\\[ A = \\sqrt{200} \\]\n","\\[ A = 10\\sqrt{2} \\]\n","\n","So, the area of the triangle is \\( 10\\sqrt{2} \\) square units.\n"]}]},{"cell_type":"markdown","source":["### Audio Summary with GPT-4o"],"metadata":{"id":"4kAhoyBWfH9x"}},{"cell_type":"code","source":["# Transcribe the audio\n","transcription = portkey.audio.transcriptions.create(\n","    model=\"whisper-1\",\n","    file=open('/content/Sample_Audio.mp3', \"rb\"),\n",")\n","\n","\n","response = portkey.chat.completions.create(\n","    model=\"gpt-4o\",\n","    messages=[\n","    {\"role\": \"system\", \"content\":\"\"\"You are generating a transcript summary. Create a summary of the provided transcription. Respond in Markdown.\"\"\"},\n","    {\"role\": \"user\", \"content\": [\n","        {\"type\": \"text\", \"text\": f\"The audio transcription is: {transcription.text}\"}\n","        ],\n","    }\n","    ],\n","    temperature=0,\n",")\n","print(response.choices[0].message.content)"],"metadata":{"id":"uc9-rtyWTRYz"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"HvxiTJcxDUCN"},"source":["## With Portkey Client"]},{"cell_type":"markdown","metadata":{"id":"04Rw2w74DXg5"},"source":["Alternatively, you can use Portkey Client instead of OpenAI.\n","\n","Note: You can safely store your OpenAI API key in [Portkey](https://app.portkey.ai/) and access models using virtual key\n"]},{"cell_type":"code","execution_count":7,"metadata":{"id":"5-TaHCg9DHtI","executionInfo":{"status":"ok","timestamp":1715680111736,"user_tz":-330,"elapsed":2058,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}}},"outputs":[],"source":["from portkey_ai import Portkey\n","\n","portkey = Portkey(\n","    api_key = userdata.get('PORTKEY_API_KEY'),   # replace with your Portkey API key\n","    virtual_key= \"gpt3-8070a6\",   # replace with your virtual key for OpenAI AI\n",")"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2845,"status":"ok","timestamp":1715621642721,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"},"user_tz":-330},"id":"H3O6_690DjR5","outputId":"6d300346-d0b9-4808-dc00-9897fd4ddea6"},"outputs":[{"output_type":"stream","name":"stdout","text":["{\n","    \"id\": \"chatcmpl-9OTqjoQ9CDChpjCVOSwh3DbJe00z7\",\n","    \"choices\": [\n","        {\n","            \"finish_reason\": \"stop\",\n","            \"index\": 0,\n","            \"logprobs\": null,\n","            \"message\": {\n","                \"content\": \"I'm an artificial intelligence developed by OpenAI, known as ChatGPT. I can help answer questions, provide information, and engage in various types of conversations. How can I assist you today?\",\n","                \"role\": \"assistant\",\n","                \"function_call\": null,\n","                \"tool_calls\": null\n","            }\n","        }\n","    ],\n","    \"created\": 1715621641,\n","    \"model\": \"gpt-4o-2024-05-13\",\n","    \"object\": \"chat.completion\",\n","    \"system_fingerprint\": \"fp_729ea513f7\",\n","    \"usage\": {\n","        \"prompt_tokens\": 11,\n","        \"completion_tokens\": 38,\n","        \"total_tokens\": 49\n","    }\n","}\n"]}],"source":["completion = portkey.chat.completions.create(\n","    messages= [{ \"role\": 'user', \"content\": 'Who are you?'}],\n","    model= 'gpt-4o'\n",")\n","\n","print(completion)"]},{"cell_type":"markdown","source":["## Advanced Routing - Load Balancing across multiple API keys"],"metadata":{"id":"GajylwyNkIDc"}},{"cell_type":"markdown","source":["With load balancing, you can distribute load effectively across multiple API keys or providers based on custom weights to ensure high availability and optimal performance.\n","\n","Let's take an example where we might want to split traffic between OpenAI's `llama-3-70b` and OpenAI's `gpt-3.5` giving a weightage of 70-30.\n","\n","The gateway configuration for this would look like the following:"],"metadata":{"id":"p1Cod5TuoSql"}},{"cell_type":"code","source":["config = {\n","  \"strategy\": {\n","      \"mode\": \"loadbalance\"\n","    },\n","  \"targets\": [\n","    {\n","      \"provider\": \"openai\",\n","      \"api_key\": \"sk-***\"\n","    },\n","    {\n","      \"provider\": \"openai\",\n","      \"api_key\": \"sk-***\"\n","    }\n","  ]\n","}"],"metadata":{"id":"JMhxoycAkNBp"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["from openai import OpenAI\n","from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n","from google.colab import userdata\n","\n","client = OpenAI(\n","    api_key=\"X\",\n","    base_url=PORTKEY_GATEWAY_URL,\n","    default_headers=createHeaders(\n","        api_key=userdata.get(\"PORTKEY_API_KEY\"),\n","        config=config\n","    )\n",")\n","\n","chat_complete = client.chat.completions.create(\n","    model=\"gpt-4o\",\n","    messages=[{\"role\": \"user\",\n","               \"content\": \"Just say hi!\"}],\n",")\n","\n","print(chat_complete.choices[0].message.content)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"EppbDt5llNsn","executionInfo":{"status":"ok","timestamp":1714222465539,"user_tz":-330,"elapsed":4496,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}},"outputId":"49c0003d-2bf9-4daa-d400-99a4d2c76537"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["gpt-3.5-turbo-0125\n","Hi! How can I assist you today?\n"]}]},{"cell_type":"markdown","metadata":{"id":"GwQmjjr0GePo"},"source":["## Observability with Portkey"]},{"cell_type":"markdown","metadata":{"id":"344KJaH3GfG_"},"source":["By routing requests through Portkey you can track a number of metrics like - tokens used, latency, cost, etc.\n","\n","Here's a screenshot of the dashboard you get with Portkey!\n","\n","![portkey_view.JPG]()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"qFCSC4GkJd2S"},"outputs":[],"source":[]}],"metadata":{"colab":{"provenance":[{"file_id":"1-yKXZ8Bdt5azMldeRvud4MQN61DRfr3C","timestamp":1715621454932}]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}